Neural Novel Actor: Learning a Generalized Animatable Neural Representation for Human Actors

We propose a new method for learning a generalized animatable neural human representation from a sparse set of multi-view imagery of multiple persons. The learned representation can be used to synthesize novel view images of an arbitrary person from a sparse set of cameras, and further animate them with the user's pose control. While existing methods can either generalize to new persons or synthesize animations with user control, none of them can achieve both at the same time. We attribute this accomplishment to the employment of a 3D proxy for a shared multi-person human model, and further the warping of the spaces of different poses to a shared canonical pose space, in which we learn a neural field and predict the person- and pose-dependent deformations, as well as appearance with the features extracted from input images. To cope with the complexity of the large variations in body shapes, poses, and clothing deformations, we design our neural human model with disentangled geometry and appearance. Furthermore, we utilize the image features both at the spatial point and on the surface points of the 3D proxy for predicting person- and pose-dependent properties. Experiments show that our method significantly outperforms the state-of-the-arts on both tasks. The video and code are available at https://talegqz.github.io/neural_novel_actor

Delving into Discrete Normalizing Flows on SO(3) Manifold for Probabilistic Rotation Modeling

Normalizing flows (NFs) provide a powerful tool to construct an expressive distribution by a sequence of trackable transformations of a base distribution and form a probabilistic model of underlying data. Rotation, as an important quantity in computer vision, graphics, and robotics, can exhibit many ambiguities when occlusion and symmetry occur and thus demands such probabilistic models. Though much progress has been made for NFs in Euclidean space, there are no effective normalizing flows without discontinuity or many-to-one mapping tailored for SO(3) manifold. Given the unique non-Euclidean properties of the rotation manifold, adapting the existing NFs to SO(3) manifold is non-trivial. In this paper, we propose a novel normalizing flow on SO(3) by combining a Mobius transformation-based coupling layer and a quaternion affine transformation. With our proposed rotation normalizing flows, one can not only effectively express arbitrary distributions on SO(3), but also conditionally build the target distribution given input observations. Extensive experiments show that our rotation normalizing flows significantly outperform the baselines on both unconditional and conditional tasks.Comment: CVPR 202

Single-molecule real-time sequencing of the full-length transcriptome of Portunus pelagicus

Reconstruction and annotation of transcripts, particularly for a species without reference genome, plays a critical role in gene discovery, investigation of genomic signatures, and genome annotation in the pre-genomic era. This is the first study to use Single-molecule real-time (SMRT) sequencing for reporting the full-length transcriptome of Portunus pelagicus. Overall, 16.26 Gb of raw reads were obtained, including 7,068,387 subreads, with average length of 2,300 bp and N50 length of 3,594 bp. In total, 351,870 circular consensus sequences (CCS) reads were extracted, including 255,378 full-length non-chimeric (FLNC) reads with mean length of 3,423 bp.70,407 genes were obtained after eliminating redundant sequences, and 56,557 (80.33%) genes were annotated in at least one database, 17,267 (24.52%) genes were annotated in all of the seven databases. Further, 68,797 coding sequences (CDS) were identified, including 36,848 complete CDS. A total of 1,730 unigenes were predicted to be transcription factors (TFs). Finally, 11,894 long noncoding RNA (lncRNA) transcripts were predicted by different computational approaches and 147,262 single sequence repeat (SSR)s were obtained. The transcriptome data reported herein are bound to serve as a basis for future studies on P. pelagicus

Learning Controllable 3D Diffusion Models from Single-view Images

Diffusion models have recently become the de-facto approach for generative modeling in the 2D domain. However, extending diffusion models to 3D is challenging due to the difficulties in acquiring 3D ground truth data for training. On the other hand, 3D GANs that integrate implicit 3D representations into GANs have shown remarkable 3D-aware generation when trained only on single-view image datasets. However, 3D GANs do not provide straightforward ways to precisely control image synthesis. To address these challenges, We present Control3Diff, a 3D diffusion model that combines the strengths of diffusion models and 3D GANs for versatile, controllable 3D-aware image synthesis for single-view datasets. Control3Diff explicitly models the underlying latent distribution (optionally conditioned on external inputs), thus enabling direct control during the diffusion process. Moreover, our approach is general and applicable to any type of controlling input, allowing us to train it with the same diffusion objective without any auxiliary supervision. We validate the efficacy of Control3Diff on standard image generation benchmarks, including FFHQ, AFHQ, and ShapeNet, using various conditioning inputs such as images, sketches, and text prompts. Please see the project website (\url{https://jiataogu.me/control3diff}) for video comparisons.Comment: work in progres

A novel SETD2 variant causing global development delay without overgrowth in a Chinese 3-year-old boy

Background: Luscan-Lumish syndrome is characterized by macrocephaly, postnatal overgrowth, intellectual disability (ID), developmental delay (DD), which is caused by heterozygous SETD2 (SET domain containing 2) mutations. The incidence of Luscan-Lumish syndrome is unclear. The study was conducted to provide a novel pathogenic SETD2 variant causing atypical Luscan-Lumish syndrome and review all the published SETD2 mutations and corresponding symptoms, comprehensively understanding the phenotypes and genotypes of SETD2 mutations.Methods: Peripheral blood samples of the proband and his parents were collected for next-generation sequencing including whole-exome sequencing (WES), copy number variation (CNV) detection and mitochondrial DNA sequencing. Identified variant was verified by Sanger sequencing. Conservative analysis and structural analysis were performed to investigate the effect of mutation. Public databases such as PubMed, Clinvar and Human Gene Mutation Database (HGMD) were used to collect all cases with SETD2 mutations.Results: A novel pathogenic SETD2 variant (c.5835_c.5836insAGAA, p. A1946Rfs*2) was identified in a Chinese 3-year-old boy, who had speech and motor delay without overgrowth. Conservative analysis and structural analysis showed that the novel pathogenic variant would loss the conserved domains in the C-terminal region and result in loss of function of SETD2 protein. Frameshift mutations and non-sense mutations account for 68.5% of the total 51 SETD2 point mutations, suggesting that Luscan-Lumish syndrome is likely due to loss of function of SETD2. But we failed to find an association between genotype and phenotype of SETD2 mutations.Conclusion: Our findings expand the genotype-phenotype knowledge of SETD2-associated neurological disorder and provide new evidence for further genetic counselling

Improving Utility of GPU in Accelerating Industrial Applications with User-centred Automatic Code Translation

SMEs (Small and medium-sized enterprises), particularly those whose business is focused on developing innovative produces, are limited by a major bottleneck on the speed of computation in many applications. The recent developments in GPUs have been the marked increase in their versatility in many computational areas. But due to the lack of specialist GPU (Graphics processing units) programming skills, the explosion of GPU power has not been fully utilized in general SME applications by inexperienced users. Also, existing automatic CPU-to-GPU code translators are mainly designed for research purposes with poor user interface design and hard-to-use. Little attentions have been paid to the applicability, usability and learnability of these tools for normal users. In this paper, we present an online automated CPU-to-GPU source translation system, (GPSME) for inexperienced users to utilize GPU capability in accelerating general SME applications. This system designs and implements a directive programming model with new kernel generation scheme and memory management hierarchy to optimize its performance. A web-service based interface is designed for inexperienced users to easily and flexibly invoke the automatic resource translator. Our experiments with non-expert GPU users in 4 SMEs reflect that GPSME system can efficiently accelerate real-world applications with at least 4x and have a better applicability, usability and learnability than existing automatic CPU-to-GPU source translators